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affective outcomes. The results were consistent when academic level and overall course performance were 
controlled. Implications and future research directions are discussed. 
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Abstract 

This study examined the performance differences in cognitive and affective outcomes 
between students who completed a short-term service-learning experience and students 
who watched a video exemplar of the task. Performance measures were obtained via a 
written reflection paper that contained both cognitive and affective components. Results 
revealed no significant differences on cognitive outcomes and significant differences on 
affective outcomes. The results were consistent when academic level and overall course 
performance were controlled. Implications and future research directions are discussed. 
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Introduction 

Service-learning opportunities in higher education have grown over the past few years, 
and this trend is expected to continue (Bringle & Hatcher, 2010; Butin, 2006; Toncar, Reid, 
Burns, Anderson, & Nguyen, 2006). Many institutions are committed to the idea of 
community engagement and have pledged to expand opportunities for students. One 
reason for the increased focus on service-learning is the support both in principle and 
practice found within the research literature. There is evidence that service-learning 
provides significant benefits to the stakeholders involved (Lester, Tomkovick, Wells, Flunker, 
& Kickul, 2005), and this theme is consistent across disciplines and pedagogy. Within the 
education discipline, for example, Root (2002) focused on teacher preparation, and showed 
that service-learning led to significant gains on measures such as acceptance of 
diversity and belief in the importance of teachers' ability to bring about social change. 
Humanities scholars Bernacki and Jaeger (2008) examined service-learning effects on moral 
development and orientation, and although there were no group differences on measured 
scores, students reported increases in compassion and sensitivity. In the business area, 
Govekarand Rishi (2007) noted the transformational potential of service-learning for 
students in management and economics courses, while McCarthy and Tucker (2002) 
discussed the benefits of service-learning in business courses commenting that it "provides 
business faculty members with the modus operandi to enrich theory with application while 
increasing community service participation" (p. 645). Reynolds and Ahern-Dodson (2010), 
who looked at service-learning in the sciences, found benefits in this pedagogy for students, 
faculty, the community, and the university. From their perspective, the research-focused 
service-learning provided a "useful strategy for promoting science literacy" (p. 24). The 
literature overwhelmingly favors service-learning, and although the mounting support will 
lead to increased and enhanced service-learning opportunities, it will also encourage closer 
scrutiny of the principle and the practice. 
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Service-learning has been defined in several places and at different times, for example, 
Bringle and Hatcher (1995), Hunter and Brisbin (2000), and Bringle and Hatcher (2009). 
Although the definitions differ slightly, the key tenets of service, reflection, and learning are 
preserved in each case. Service-learning can be described as an academic activity where: 
the service rendered is a benefit to both the community and the student; the reflection 
process provides an opportunity for synthesizing the experience and applying it to course 
content; and the learning is educationally meaningful, thus, the service-learning task is 
anchored to educational objectives. All three tenets are important, but this research study 
is focused primarily on reflection and learning. 

The reflection process is a critical part of the service-learning experience because student 
reflections "bridge the community service activities and the educational content of the 
course in a way that produces new learning not delivered solely by the course content or 
the community service alone" (Bringle & Hatcher, 2010, p. 5). The importance of reflection 
is clearly articulated in the literature. Bringle (1999) explicitly addressed the issue of 
reflection and used Deweyan thought as a philosophical basis for advancing reflection as an 
integral component of the service-learning experience. The importance of reflection is also 
highlighted by Dubinsky's (2006) discussion of the role of reflection when creating 
assignments, Hatcher, Bringle, and Muthiah's (2004) examination of the nature and quality 
of reflection, and Welch and James's (2007) investigation of a guided reflection technique. 
Ash and Clayton (2009) viewed reflection as a central figure in the learning process and 
proposed that reflection becomes "critical reflection" (p. 27) when it functions as a 
metacognitive process designed to improve thought and action, and the relationship 
between the two. Within their model, intentional, well-designed, and purposeful reflection 
generates, deepens, and documents learning. 

The reflection process is primarily a benefit for students, but it may also be a useful tool for 
faculty. Although surveys have traditionally been the means of studying the effects of 
service-learning, where Lester et al. (2005) and Pierrakos, Borrego, and Lo (2007) are two 
examples, the reflection process may also provide faculty with an excellent assessment 
opportunity. Reflection as an assessment tool is becoming increasingly prevalent in the 
literature as exemplified by Litke's (2002) analysis of student reflection papers and Molee, 
Henry, Sessa, and McKinney-Prupis's (2010) model that assessed student learning using 
reflection. Similarly, Ash, Clayton, and Atkinson (2005) demonstrated that written 
reflection could be used as an assessment tool, and they also demonstrated that it could 
be used to assess cognitive outcomes. 

The learning aspect of service-learning has been the focus of some criticism. The literature 
is clear and consistent regarding attitudinal shifts and other personal gains (Eyler & Giles, 
1999; Toncar et al., 2006) but one outstanding question is whether or not students can 
identify, extract, and interpret relevant academic content from service-learning experiences 
(Eyler, 2000; Steinke, Fitch, Johnson, & Waldstein, 2002). In essence, the question is 
whether or not students perform better academically as a result of the service-learning 
process. Casile, Hoover, and O'Neil (2011) focused on cognitive outcomes and 
demonstrated that service-learning could enhance content mastery in a management 
course. Additionally, Yorio and Le's (2012) meta-analysis revealed that service-learning had 
a positive effect on both cognitive and affective outcomes. Although there is accumulating 
evidence that service-learning can be linked to gains in cognitive outcomes, assessing 
cognitive outcomes using student written reflection continues to be a developing area of 
interest in the field. Consistent with the evolving literature, for example Ash et al.'s (2005) 
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call to "document the academic and cognitive outcomes produced by the pedagogy" (p. 49) 
and Bringle and Hatcher's (2009) recommendation to incorporate written narratives "as 
authentic evidence that is collected through a structured reflection process" (p. 44), the 
current research study investigated the contributions of service-learning experiences to 
cognitive and affective outcomes using students' written reflection papers. 

Time is an important consideration in service-learning, but little research has been done in 
the area. In one study, community partners identified time as one of the major challenges 
in terms of training and orientation, coordinating placements, and working with student 
volunteers (Birdsall, 2005). According to Daynes and Longo (2004), many of the issues 
surrounding time are rooted in the way institutions perceive, construct, and use time as a 
resource. They noted the difficulty in maintaining long-term community partnerships when 
students complete mainly short-term service-learning experiences. The sentiment was 
echoed by Tryon et al. (2008) who also focused on the issue of time and the effects of 
short-term service-learning. They presented strategies that could help community partners 
maximize short-term experiences, and they challenged institutions to increase focus on the 
long-term needs of their community partners. Although long-term experiences are 
preferred, there is value to short-term experiences. For example, Reed, Christian Jernstedt, 
Hawley, Reber, and DuBois's study involving students who visited dying patients showed 
that "even a minimal service-learning experience can have measurable impacts on the 
outlook and attitudes of student participants" (2005, p. 367). 

The current research study was conducted within the context of a pilot project to determine 
the feasibility of integrating a service-learning component into a large-enrollment course. 
The purpose of the pilot project was not only to test the logistics of the integration, but to 
also begin the investigation into the cognitive and affective benefits associated with this 
service-learning offering. There were only a limited number of placements due to capacity 
restrictions outside the researcher's control, hence the decision to create an exemplar video 
for the remaining students. The video format was chosen because of its demonstrated 
pedagogical value (see Cox et al., 2006 for an example). In the interest of fairness, there 
was no opportunity to implement a control group where some students would not be 
exposed to any type of experience. 

The purpose of the current research study was to investigate the differences in cognitive 
and affective outcomes between students who completed a short-term service-learning 
experience and students who watched a video exemplar of the task. The comparison was 
based on students' performance scores on a written reflection paper that contained both 
cognitive and affective components. Additionally, the study examined the potential 
performance differences while controlling for academic level (freshman, sophomore, junior, 
senior) and the final letter grade that was earned in the course. In this study, cognitive 
outcome was operationalized as the performance scores received on the cognitive 
components of the reflection paper. The cognitive components were discrete opportunities 
where students could demonstrate understanding and synthesis of specific course content. 
Similarly, affective outcome was operationalized as the performance scores received on 
the affective components of the reflection paper. The affective components were discrete 
opportunities where students could report their feeling and perceptions. 

The questions explored were as follows: 

1. Are there differences in cognitive outcomes between students who completed a 

service-learning experience and students who watched a video exemplar of the task? 
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2. Are there differences in affective outcomes between students who completed a 

service-learning experience and students who watched a video exemplar of the task? 


Method 


Participants 

The participants were 263 undergraduate students from a large mid-western university. 

All students were enrolled in an introductory-level educational psychology course. The 
students' academic levels ranged from freshmen to seniors, and they were from diverse 
academic backgrounds. Sixty-eight percent of the students majored in one of the education 
areas. The study had Institutional Review Board (IRB) approval. 

Measures and Materials 

Video exemplar of the task. The researcher produced a video exemplar of the service¬ 
learning task depicting an actual reading and interaction session based on the "First Book" 
program, which provides books to children. The session involved a female volunteer 
undergraduate student and a six-year-old female child. The student was a senior education 
major who had training in, and exposure to tasks and conditions similarly to those in this 
study. The child was the daughter of a graduate student who was not connected to the 
researcher. The child was within the same age range as the children who interacted with 
the participants who completed the actual service-learning experience. The student and the 
child were from different ethnic backgrounds, and they had never met prior to the session. 

A senior faculty member who specialized in organizing service opportunities and training 
volunteers trained the student prior to the recorded session. The general training topics 
included: the anticipated sequence of events, expectations, interacting with young children, 
asking and answering questions, being supportive, and being engaging, to name a few 
topics. The faculty member presented potential scenarios, modeled appropriate responses, 
practiced with the student, and then discussed the results with the student. Some training 
time was devoted to techniques for embedding course content within the interaction. 
Specifically, the student was trained to guide the interaction such that cognitive, language, 
socioemotional, and diversity issues would emerge. For example, socioemotional artifacts 
may emerge if a child is prompted to share her or his feelings on a certain topic or if they 
describe specific experiences. The faculty member worked with the student to ensure that 
this and other content-specific lines of interaction would occur and thus be directly 
observable by those who watched the video. After completing the general training session, 
the volunteer student rehearsed potential scenarios with a graduate student who had 
volunteered to assist in the training. This practice session provided a final opportunity for 
the student to focus on creating a meaningful experience for the child, and also interacting 
in a purposeful manner such that specific course content would emerge during the session. 

On the day of taping, the faculty member introduced the student and the child and 
remained in the room to observe the interaction. The student and the child read two age- 
appropriate books, after which the student gave the books to the child as a gift. The child 
then wrote a "thank you" note, a requirement of the "First Book" program, after which she 
returned to her parent. The faculty member and the student then debriefed the interaction 
session reflecting on both specific events that occurred and the general overall experience. 
The entire session, from introduction of the participant to the debriefing of the experience, 
was video recorded. 
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The session provided a wealth of observable artifacts covering the intended content areas 
and much more. The richness of the interaction was probably the result of the student's 
competence, the training, and the child's personality and verbal nature. Specific content 
concepts, for example egocentric behavior, were directly observable in the interaction while 
other concepts such as those related to gender and diversity could be inferred based on the 
child's reactions and stated preferences. The interaction and subsequent video provided 
ample opportunities for both direct and indirect observation of key course concepts 
occurring under somewhat controlled circumstances. 

The video recording was edited to highlight important events that occurred during the 
session. The final edited version of the video was thirty minutes and twenty-one seconds in 
length. It contained an introduction to the task, selected portions of the practice session 
with the graduate student, the entire reading session, the post-discussion between the 
student and the faculty member, and a summary review of the task. As a part of the editing 
process, the video was paused before important events occurred, and subtitles were added 
to introduce those events. The subtitles also contained recommendations and tips that 
would make the experience meaningful for both adult and child. 

Document exemplar of the task. The researcher created an exemplar document outlining 
the task, expectations, and tips for meaningful interaction to name a few topics. This 
document contained a textual version of all the training materials that were used, and all 
the subtitles, hints, and review notes from the video exemplar of the task. The document, 
in essence, contained all the information presented during training and those included on 
the video. It was not feasible for the faculty member to train all the participants who would 
complete the actual service-learning experience. This document, however, provided a 
structure and method for those participants to embed course content into their interactions 
and to recognize course constructs as they emerged from the interactions. 

Instruments 

Pre-reflection paper. Each participant in the study completed two written papers. The first 
paper, a pre-reflection, was brief and required students to envision what it would be like to 
conduct a reading session with a child between the ages of five and seven. The pre¬ 
reflection had two parts titled The experience and Connecting the content and your 
experience. The first part, The experience, comprised two sections and focused on the 
reading experience from the perspectives of (a) the adult and (b) the child. Students were 
asked to estimate the current and future effects that a reading session with a child would 
have on first, himself or herself and second, the child. The second part, Connecting the 
content and your experience, comprised four sections and focused on extracting academic 
content from the experience. The four sections represented four distinct content areas: (a) 
cognitive development, (b) language development, (c) socioemotional development, and (d) 
diversity. For each section, students were asked to describe what they expected to observe 
from a child between the ages of five and seven, and to specify exactly what they would 
look for in the interaction to support their conclusions. The pre-reflection was a course 
requirement, but it was not graded. 

Reflection paper. The final written reflection paper was a course requirement and it was 
graded. The reflection paper had three parts titled The experience, Connecting the content 
and your experience, and Comparing pre-reflection and reflection. The first two parts were 
very similar to those described in the pre-reflection, with the only difference being that the 


https://doi.org/ 1 0.2042 9/ij SO tl.20 1 3.0702 15 


5 



Cognitive and Affective Outcomes of Short-Term 


sections were worded in the past tense, and students were required to describe their actual 
observations and the specific mechanisms they used to support their conclusions. The third 
part, Comparing pre-reflection and reflection, required students to compare their original 
perceptions described in the pre-reflection paper to their perceptions after the service or 
video experience. Students were asked to highlight similarities and differences and provide 
supported rationales for the differences. The first and third parts, The experience and 
Comparing pre-reflection and reflection, were used to determine attitude outcomes because 
they presented students with opportunities to reflect on their feeling and perceptions. The 
second part, Connecting the content and your experience, was used to determine cognitive 
outcomes because it presented students with the opportunity to demonstrate understanding 
and synthesis of the four content areas. 

Procedures 

During the first week of the semester, students in the course were divided into groups of five 
or six for pedagogical reasons not related to service-learning. Each student then completed 
the p re-reflect ion paper that was due at the end of the fourth week of the semester. One 
student from each group was then randomly selected to participate in the actual service¬ 
learning experience while the others in the group watched the video exemplar of the activity. 
This resulted in 51 students completing the actual service-learning experience while 212 
students watched the video exemplar of the task. The number of students selected to 
complete the actual service-learning experience was a function of the capacity of the service¬ 
learning program at the time. 

The service and video activities were conducted during the ninth week of the semester. 
Students who completed the experience were required to read the exemplar document 
before they were assigned to a service location. Each of the 51 service-learning students 
then collected two books, which were the same books used in the video exemplar, and then 
traveled to a local school. At the school, each student was assigned a young reader, 
between the ages of five and seven, based on the school's need for that day. After 
introductions were completed, the student and the child were assigned a quiet reading area. 
Although each experience had unique elements, each student spent time reading and 
interacting with a child, and then each presented the books to the child as gifts. The 
remaining 212 students watched the video exemplar of the task during the same time 
frame. All 263 students then wrote reflection papers based on the previously described 
criteria. 

Data Preparation and Analysis 

All papers were initially graded by the two teaching assistants (TA) that were assigned to the 
course. Both TAs were graduate students with extensive knowledge in the course content 
areas, and they were both trained by the course instructor. A different graduate assistant 
(GA) who was not affiliated with the course also graded the papers, but these grades were 
for research purposes only. In essence, each paper was graded by one of the TAs and also 
by the GA. The GA had completed graduate work in the content areas and was also trained 
by the course instructor prior to grading. The course instructor conducted one training 
session with the GA and it was identical to the training that the two TAs received. The 
training session involved reviewing general and specific grading policies and procedures, 
discussing the grading categories, and clarifying the criteria for correct and incorrect 
responses for the cognitive components. The final training task was grading and then 
reviewing five sample papers. 
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All grading was done using a spreadsheet format where graders would assign numeric 
scores to each section of a paper. For example, a paper would receive one score for the 
Cognitive Development section and a separate score for the Diversity section. The data 
revealed high agreement between the total scores given by each TA and those given by the 
GA. Intraclass correlation coefficients (ICC) were computed at the 95% confidence interval 
for the total scores given to the papers. The first TA graded 153 papers and the ICC was 
.80 (p < .001). The second TA graded 110 papers and the ICC was .76 (p < .001). These 
data confirmed the reliability of the grades given by the GA. 

The research questions were answered using the Multivariate Analysis of Variance 
(MANOVA) procedure. The conservative Pillai's Trace was preferred to the Wilks' Lambda 
because of its robustness to unbalanced designs. Additional analyses using the Multivariate 
Analysis of Covariance (MANCOVA) procedure were conducted to control for the possible 
effects of academic level (freshman, sophomore, junior, senior) and final course letter 
grade. 


Results 

Focusing on the first research question that explored cognitive effects, the data revealed 
no significant differences in students' performance scores on the cognitive outcomes of 
the reflection paper. Table 1 shows the full results for the multivariate analysis. 


Table 1 

Multivariate Effects of Experience on Outcomes _ 

Partial 


Test 

Value 

F 

Hypothesis 

df 

Error df 

Sig. 

Eta 

Squared 

Observed 

Power 

Cognitive Outcomes 

Pillai's Trace 

.022 

1.435 

4.000 

258.000 

.223 

.022 

.444 

Wilks' Lambda 

.978 

1.435 

4.000 

258.000 

.223 

.022 

.444 

Hotelling's Trace 

.022 

1.435 

4.000 

258.000 

.223 

.022 

.444 

Roy's Largest Root 

.022 

1.435 

4.000 

258.000 

.223 

.022 

.444 

Affective Outcomes 

Pillai's Trace 

.092 

8.695 

3.000 

259.000 

<.001 

.092 

.994 

Wilks' Lambda 

.908 

8.695 

3.000 

259.000 

<.001 

.092 

.994 

Hotelling's Trace 

.101 

8.695 

3.000 

259.000 

<.001 

.092 

.994 

Roy's Largest Root 

.101 

8.695 

3.000 

259.000 

<.001 

.092 

.994 


A one-way MANOVA at the .05 level revealed no significant multivariate effect between the 
two groups in terms of performance scores on the content knowledge sections of the 
reflection paper, Pillai's Trace = .022, F( 4, 258) = 1.435, p = .223, observed power = .444. 
Box's M = 23.737 was not significant (F = 2.302, p = .011) at the .005 level as 
recommended by Huberty and Petoskey (2000), thus upholding the assumption of 
homoscedasticity. Bartlett's Test of Sphericity was significant (p < .001) thus the 
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appropriateness of the MANOVA. The Levene's Test resulted in the following significance 
levels for the cognitive components: Cognitive = .701, Language = .266, Socioemotional = 
.327, and Diversity = .687, upholding the assumption of equal group error variances. Table 
2 shows the performance on the cognitive outcomes for students who completed the 
service-learning experience and those who watched the video exemplar of the task. 


Table 2 

Students' Performance on the Cognitive Components (Cognitive Outcomes) 


Cognitive Components 

n 

Experience 

M SD 

n 

Video 

M 

SD 

Cognitive Development 

51 

8.18 

0.817 

212 

8.27 

1.031 

Language Development 

51 

8.02 

1.029 

212 

7.98 

1.297 

Socioemotional Development 

51 

7.98 

1.140 

212 

7.71 

1.287 

Diversity 

51 

7.63 

1.166 

212 

7.75 

1.160 


Note. The maximum score for each outcome was nine points. 


The possible confounding effects of students' academic level and final course letter grade 
were considered, but neither influenced the non-significant results on the cognitive 
outcomes. Table 3 shows the full results for the multivariate analysis with covariates. 


Table 3 

Multivariate Effects of Experience on Outcomes with Covariates 


Test 

Value 

F 

Hypothesis 

df 

Error df 

Sig. 

Partial 

Eta 

Squared 

Observed 

Power 



Cognitive Outcomes 




Academic Level Covariate 








Pillai's Trace 

.022 

1.430 

4.000 

257.000 

.224 

.022 

.442 

Wilks' Lambda 

.978 

1.430 

4.000 

257.000 

.224 

.022 

.442 

Hotelling's Trace 

.022 

1.430 

4.000 

257.000 

.224 

.022 

.442 

Roy's Largest Root 

.022 

1.430 

4.000 

257.000 

.224 

.022 

.442 

Final Course Letter Grade 

Covariate 







Pillai's Trace 

.023 

1.538 

4.000 

257.000 

.192 

.023 

.473 

Wilks' Lambda 

.977 

1.538 

4.000 

257.000 

.192 

.023 

.473 

Hotelling's Trace 

.024 

1.538 

4.000 

257.000 

.192 

.023 

.473 

Roy's Largest Root 

.024 

1.538 

4.000 

257.000 

.192 

.023 

.473 
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Affective Outcomes 

Academic Level Covariate 


Pillai's Trace 

.092 

8.681 

3.000 

258.000 

<.001 

.092 

.994 

Wilks' Lambda 

.908 

8.681 

3.000 

258.000 

<.001 

.092 

.994 

Hotelling's Trace 

.101 

8.681 

3.000 

258.000 

<.001 

.092 

.994 

Roy's Largest Root 

.101 

8.681 

3.000 

258.000 

<.001 

.092 

.994 

Final Course Letter Grade 

Covariate 







Pillai's Trace 

.095 

9.002 

3.000 

258.000 

<.001 

.095 

.996 

Wilks' Lambda 

.905 

9.002 

3.000 

258.000 

<.001 

.095 

.996 

Hotelling's Trace 

.105 

9.002 

3.000 

258.000 

<.001 

.095 

.996 

Roy's Largest Root 

.105 

9.002 

3.000 

258.000 

<.001 

.095 

.996 


The MANCOVAs conducted at the .05 level revealed no significant multivariate effects when 
controlling for academic level (Pillai's Trace = .022, F(4, 257) = 1.430, p = .224, observed 
power = .442) or final course letter grade (Pillai's Trace = .023, F( 4, 257) = 1.538, p = 
.192, observed power = .473). 

For the second research question, which focused on affective outcomes, the data revealed 
significant differences in students' performance scores on the affective outcomes of the 
reflection paper. A one-way MANOVA at the .05 level (see Table 1 for full results) revealed 
significant multivariate effects between the two groups in terms of performance scores on 
the affective sections of the reflection paper, Pillai's Trace = .092, F( 3, 259) = 8.70, p < 

.05, n P 2 = .092, observed power = .994. The observed power was excellent, and it was well 
above the .80 desired level. This indicated an increased probability of correctly detecting an 
existing effect, and a decreased risk of a type II error. Box's M = 18.971 was significant (F 
= 3.090, p = .005) at the recommended .005 level, indicating an increased risk of a type I 
error, but it was considered acceptable because of the high power result. Bartlett's Test of 
Sphericity was significant (p < .001) thus the appropriateness of the MANOVA. The 
Levene's Test resulted in the following significance levels for the affective components: 
Personal Perspective = .350, Child's Perspective = .338, Comparison = .846, upholding the 
assumption of equal group error variances. Table 4 shows the performance on the affective 
outcomes for students who completed the service-learning experience and those who 
watched the video exemplar of the task. 


Table 4 

Students' Performance on the Affective Components (Affective Outcomes) 


Affective Components 


Service 


Video 


n 

M 

SD 

n 

M 

SD 

Personal Perspective 

51 

8.53 

0.703 

212 

7.95 

0.867 

Child's Perspective 

51 

8.29 

0.879 

212 

7.82 

0.820 

Comparison 

51 

14.31 

0.836 

212 

14.18 

1.344 


Note. The maximum score for the Comparison outcome was sixteen points. The maximum for all other outcomes 
was nine points. 
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Subsequent univariate tests revealed significant differences on Personal Perspective F( 1, 
261) = 19.798, p < .05, q p 2 = .071, observed power = .993 and Child's Perspective F( 1, 
261) = 13.593, p < .05, q p 2 = .050, observed power = .957, where, in each case, the group 
that completed the actual service-learning experience outperformed the group that watched 
the video. There were no significant differences between the groups in terms of Comparison 
F( 1, 261) = .466, p = .495, q p 2 = .002, observed power =.102. The observed power for 
Personal Perspective and Child's Perspective were excellent and well above the desired level 
of .80, but the observed power for Comparison was not. 

Neither students' academic level nor final course letter grade influenced the performance 
results on the affective outcomes. The MANCOVAs conducted at the .05 level (see Table 3 
for full results) revealed that the significant multivariate effects between the two groups 
were still present when the data were controlled for academic level (Pillai's Trace = .092, 

F( 3, 258) = 8.681, p < .05, q p 2 = .092, observed power = .994) and final course letter 
grade (Pillai's Trace = .095, F( 3, 258) = 9.002, p < .05, q p 2 = .095, observed power = 

.996). The observed power in both cases were excellent and well above the .80 desired 
level. Similar to the results obtained without the covariates, subsequent univariate tests 
revealed significant differences on Personal Perspective and Child's Perspective, but not 
Comparison. Table 5 shows the results of the univariate analysis for the affective 
outcomes. 


Table 5 

Univariate Analysis of Variance for Affective Outcomes 


Covariates 

F 

Sig. 

Partial Eta 
Squared 

Observed 

Power 

Academic Level 





Personal Perspective 

19.768 

<.001 

.071 

.993 

Child's Perspective 

13.628 

<.001 

.050 

.957 

Comparison 

0.471 

.493 

.002 

.105 

Final Course Letter Grade 





Personal Perspective 

20.279 

<.001 

.072 

.994 

Child's Perspective 

14.222 

<.001 

.052 

.964 

Comparison 

0.322 

.571 

.001 

.087 

Note. Univariate df = (1,260) 


Again, the observed power for Personal Perspective and Child's Perspective were excellent 
and well above the desired level of .80, but the observed power for Comparison was not. 
Where differences existed, the group that completed the actual service-learning experience 
outperformed the group that watched the video (see Table 4). 
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Discussion 

The results of this study on the cognitive and affective outcomes of short-term service¬ 
learning experiences provided some evidence and support for emerging perspectives, 
and they also confirmed some established views within the field. Specifically, this study 
provided some evidence of the link between service-learning experiences and positive 
cognitive outcomes. Also, this study provided some evidence of the viability of using 
written reflection to assess service-learning outcomes. In terms of established views, this 
study confirmed the positive affective outcomes that are possible via service-learning 
experiences. 

The data analysis for the question focused on students' performance on cognitive outcomes 
showed no significant differences between students who completed the service-learning 
experience and those who watched the video exemplar of the task. That is, those who 
completed the experience performed on par with those who watched the video. This finding 
is important because it suggests that students can synthesize their service-learning 
experiences and consequently connect those experiences to the requisite academic content. 
These students were able to make the connections as well as those who watched the video 
exemplar of task, which was purposefully infused with academic content. It is conceivable 
that these results were in part due to the exemplar document providing sufficient structure 
to guide the interaction. The results remained consistent when academic level and final 
course letter grade were considered. In this case, academic level could be considered a 
measure of student academic maturity, and final course letter grade could be considered a 
measure of student motivation and ability within the course. Although service-learning 
targets much more that academic outcomes, connecting experience to content is still a 
very important goal. 

The results for the second research question confirmed the effect that service-learning has 
on affective outcomes. Students who completed the service-learning experience were 
better able to describe the current and future effects of reading with a child, from both their 
personal and the child's perspective. Although watching the video could be viewed as an 
experience, Kolb (1984) outlined the importance of actual experience in the learning 
process, and this research provided support for this position. Students who had the actual 
experience were able to use it as an anchor to describe the current and future effects of the 
experience. Although the exemplar document may partially explain these results, perhaps 
there are affective constructs embedded within actual experience that are only revealed 
during the course of human interaction. Similar to the results for the first research 
question, the results remained consistent when academic level and final course letter grade 
were considered. This is particularly important because it suggested that academic 
maturity, which might be expressed in writing, did not affect the results. Similarly, there 
is no evidence that student motivation and ability contributed to the results. 

Another important implication that can be drawn from the results on the first research 
question is the successful demonstration of using written reflection as a tool to assess 
cognitive outcomes in service-learning. Students were able to identify the academic 
constructs embedded within their experiences and describe the evidence supporting their 
conclusions. This is consistent with the call in the field to increase focus on cognitive 
outcomes and to consider written reflection as a viable assessment tool (Ash et al., 2005). 
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One aspect of the study that must be noted is the relative cost of this type of assessment. 
Ash et al. (2005) mentioned the work involved in using written narratives as an assessment 
tool and this research echoes the concerns. It is a tremendous investment in time and 
resources, especially in the large-enrollment environment. Written reflections, however, 
present great potential, and they may be better measures of student learning. Another 
important aspect of the study is the issue of short-term experiences. Although long-term 
experiences are acknowledged as ideal, this study joins Reed et al. (2005) in suggesting 
that there is value in short-term service-learning experiences. 


Limitations 

This research was limited by several factors. First, this research was a part of a pilot 
project to determine the feasibility of integrating a service-learning component into a large- 
enrollment course. Therefore, those students who were selected to complete the 
experience only had one opportunity to do so. Under normal circumstances, the preferred 
format for service-learning is to have an extended experience where students experience 
more and ultimately learn more. The preferred service-learning format was not possible 
during the pilot. Next, the number of students selected to complete the service-learning 
task was a function of the number of service-learning slots available at the time. This 
resulted in unbalanced groups, and it made the statistical analyses less robust. Another 
limitation of the study involved grading. There is an acknowledged element of subjectivity 
inherent in the task of grading reflection papers. Although it is possible that grading errors 
may have occurred, the use of multiple graders, the level of training provided, and the high 
inter-rater reliability, likely mitigated the issue of grading subjectivity. Next, the current 
study controlled for the potential effects of academic level and course performance, but 
other variables may have impacted the results. Also, the exemplar document was a very 
important artifact in this study but the detail to which students attended to the document 
remains unknown. The level of student engagement with the exemplar document would 
have an effect on subsequent results. Next, although students reflected on the experience, 
their actual experiences may differ from the report due to lack of precision in recall. Finally, 
service-learning experiences can be very dynamic, and as such, these varied experiences 
may affect individual students differently. The exemplar document, however, gave students 
a guiding structure, which may have somewhat reduced the variability of student 
experiences. 


Conclusion and Future Considerations 

The current research study suggests that service-learning can be used to target cognitive 
outcomes. The emphasis on cognitive outcomes is an emerging area of interest in the field, 
and these results show that students are able to identify and synthesize the academic 
content embedded within their service-learning experiences. The results also confirm that 
service-learning can have a positive effect on affective outcomes; a finding well supported 
in the research literature. Future research could determine if these results generalize 
across academic disciplines and service-learning venues. Also, the temporal and 
management costs of service-learning are seldom examined from the perspective of the 
student. Future research should examine the cognitive and affective outcomes in reference 
to the time required for students to manage their service learning experiences. Finally, the 
current research answers a call in the research literature by demonstrating the use of 
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written reflection as a means of assessing both cognitive and affective outcomes. Additional 
research could provide insight into the best strategies for using written reflection where the 
benefits and resource cost are factored into the equation. 
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